{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4d19a50f",
   "metadata": {},
   "source": [
    "# 指令的微调"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "dcb8c267",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "numpy version: 2.0.2\n",
      "matplotlib version: 3.10.3\n",
      "tiktoken version: 0.9.0\n",
      "torch version: 2.7.1\n",
      "tqdm version: 4.67.1\n",
      "tensorflow version: 2.19.0\n"
     ]
    }
   ],
   "source": [
    "from importlib.metadata import version\n",
    "\n",
    "pkgs = [\n",
    "    \"numpy\",       # PyTorch & TensorFlow dependency\n",
    "    \"matplotlib\",  # Plotting library\n",
    "    \"tiktoken\",    # Tokenizer\n",
    "    \"torch\",       # Deep learning library\n",
    "    \"tqdm\",        # Progress bar\n",
    "    \"tensorflow\",  # For OpenAI's pretrained weights\n",
    "]\n",
    "for p in pkgs:\n",
    "    print(f\"{p} version: {version(p)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f4c15ee",
   "metadata": {},
   "source": [
    "![](./images/overview.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f5b9b57",
   "metadata": {},
   "source": [
    "## 介绍指令微调"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80719cf2",
   "metadata": {},
   "source": [
    "- 在第五章中，我们了解到预训练大语言模型涉及一个逐词生成的学习过程；\n",
    "- 因此，经过预训练的LLM擅长文本补全，但在遵循指令方面表现不佳；\n",
    "- 本章将指导LLM更好地理解并执行指令；\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e49ec089",
   "metadata": {},
   "source": [
    "![](./images/instruction-following.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1bbe7ca",
   "metadata": {},
   "source": [
    "## 构建指令微调的数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "852a0915",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of entries: 1100\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "import os\n",
    "import urllib\n",
    "\n",
    "\n",
    "def download_and_load_file(file_path, url):\n",
    "\n",
    "    if not os.path.exists(file_path):\n",
    "        with urllib.request.urlopen(url) as response:\n",
    "            text_data = response.read().decode(\"utf-8\")\n",
    "        with open(file_path, \"w\", encoding=\"utf-8\") as file:\n",
    "            file.write(text_data)\n",
    "\n",
    "    with open(file_path, \"r\", encoding=\"utf-8\") as file:\n",
    "        data = json.load(file)\n",
    "\n",
    "    return data\n",
    "\n",
    "\n",
    "file_path = \"instruction-data.json\"\n",
    "url = (\n",
    "    \"https://gitee.com/jason-dong/build-LLM-from-scratch/blob/master/section7-assistant/instruction-data.json\"\n",
    ")\n",
    "\n",
    "data = download_and_load_file(file_path, url)\n",
    "print(\"Number of entries:\", len(data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "c2891b3d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Example entry:\n",
      " {'instruction': 'Identify the correct spelling of the following word.', 'input': 'Ocassion', 'output': \"The correct spelling is 'Occasion.'\"}\n"
     ]
    }
   ],
   "source": [
    "print(\"Example entry:\\n\", data[50])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "dea3abf9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Another example entry:\n",
      " {'instruction': \"What is an antonym of 'complicated'?\", 'input': '', 'output': \"An antonym of 'complicated' is 'simple'.\"}\n"
     ]
    }
   ],
   "source": [
    "print(\"Another example entry:\\n\", data[999])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52d48670",
   "metadata": {},
   "source": [
    "![](./images/prompt-style.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0e26f68",
   "metadata": {},
   "source": [
    "- 本章采用Alpaca风格提示格式化方法，该方法最初用于指令微调的提示模板；\n",
    "- 接下来我们将对输入数据进行格式化处理，以便将其传递给大语言模型作为输入；\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "415a240c",
   "metadata": {},
   "outputs": [],
   "source": [
    "def format_input(entry):\n",
    "    instruction_text = (\n",
    "        f\"Below is an instruction that describes a task. \"\n",
    "        f\"Write a response that appropriately completes the request.\"\n",
    "        f\"\\n\\n### Instruction:\\n{entry['instruction']}\"\n",
    "    )\n",
    "\n",
    "    input_text = f\"\\n\\n### Input:\\n{entry['input']}\" if entry[\"input\"] else \"\"\n",
    "\n",
    "    return instruction_text + input_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "c7fd59d3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n",
      "\n",
      "### Instruction:\n",
      "Identify the correct spelling of the following word.\n",
      "\n",
      "### Input:\n",
      "Ocassion\n",
      "\n",
      "### Response:\n",
      "The correct spelling is 'Occasion.'\n"
     ]
    }
   ],
   "source": [
    "model_input = format_input(data[50])\n",
    "desired_response = f\"\\n\\n### Response:\\n{data[50]['output']}\"\n",
    "\n",
    "print(model_input + desired_response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "59b64c0e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n",
      "\n",
      "### Instruction:\n",
      "What is an antonym of 'complicated'?\n",
      "\n",
      "### Response:\n",
      "An antonym of 'complicated' is 'simple'.\n"
     ]
    }
   ],
   "source": [
    "model_input = format_input(data[999])\n",
    "desired_response = f\"\\n\\n### Response:\\n{data[999]['output']}\"\n",
    "\n",
    "print(model_input + desired_response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "285d1ef3",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_portion = int(len(data) * 0.85)  # 85% for training\n",
    "test_portion = int(len(data) * 0.1)    # 10% for testing\n",
    "val_portion = len(data) - train_portion - test_portion  # Remaining 5% for validation\n",
    "\n",
    "train_data = data[:train_portion]\n",
    "test_data = data[train_portion:train_portion + test_portion]\n",
    "val_data = data[train_portion + test_portion:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "ff29add6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set length: 935\n",
      "Validation set length: 55\n",
      "Test set length: 110\n"
     ]
    }
   ],
   "source": [
    "print(\"Training set length:\", len(train_data))\n",
    "print(\"Validation set length:\", len(val_data))\n",
    "print(\"Test set length:\", len(test_data))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ea17d00",
   "metadata": {},
   "source": [
    "## 组织训练批次数据"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d3be708",
   "metadata": {},
   "source": [
    "![](./images/chapter-overview-2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f6c1458",
   "metadata": {},
   "source": [
    "![](./images/detailed-batching.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1720cec2",
   "metadata": {},
   "source": [
    "- 首先，我们实现一个InstructionDataset类，该类会对数据集中的所有输入进行预标记化处理，其功能类似于第6章中的SpamDataset；"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "7c5b0400",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch.utils.data import Dataset\n",
    "\n",
    "\n",
    "class InstructionDataset(Dataset):\n",
    "    def __init__(self, data, tokenizer):\n",
    "        self.data = data\n",
    "\n",
    "        # Pre-tokenize texts\n",
    "        self.encoded_texts = []\n",
    "        for entry in data:\n",
    "            instruction_plus_input = format_input(entry)\n",
    "            response_text = f\"\\n\\n### Response:\\n{entry['output']}\"\n",
    "            full_text = instruction_plus_input + response_text\n",
    "            self.encoded_texts.append(\n",
    "                tokenizer.encode(full_text)\n",
    "            )\n",
    "\n",
    "    def __getitem__(self, index):\n",
    "        return self.encoded_texts[index]\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60a6a92b",
   "metadata": {},
   "source": [
    "- 与第6章类似，我们需要将多个训练样本汇集到批次中以加速训练过程；这要求将所有输入填充至相近长度；\n",
    "- 同样延续前一章的做法，我们采用<|endoftext|>标记作为填充标记；\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "fecca037",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[50256]\n"
     ]
    }
   ],
   "source": [
    "import tiktoken\n",
    "tokenizer = tiktoken.get_encoding(\"gpt2\")\n",
    "\n",
    "print(tokenizer.encode(\"<|endoftext|>\", allowed_special={\"<|endoftext|>\"}))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72a41fd1",
   "metadata": {},
   "source": [
    "- 在第6章中，我们将数据集中所有样本填充至相同长度；\n",
    "    - 此处我们采用更精细的方法，开发可传递给数据加载器的自定义\"归位\"函数；\n",
    "    - 该自定义归位函数将每个训练批次中的样本填充至相同长度（但不同批次可具有不同长度）；\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4859056",
   "metadata": {},
   "source": [
    "![](./images/padding.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "727d5b95",
   "metadata": {},
   "outputs": [],
   "source": [
    "def custom_collate_draft_1(\n",
    "    batch,\n",
    "    pad_token_id=50256,\n",
    "    device=\"cpu\"\n",
    "):\n",
    "    batch_max_length = max(len(item)+1 for item in batch)\n",
    "\n",
    "    # Pad and prepare inputs\n",
    "    inputs_lst = []\n",
    "\n",
    "    for item in batch:\n",
    "        new_item = item.copy()\n",
    "        # Add an <|endoftext|> token\n",
    "        new_item += [pad_token_id]\n",
    "        # Pad sequences to batch_max_length\n",
    "        padded = (\n",
    "            new_item + [pad_token_id] *\n",
    "            (batch_max_length - len(new_item))\n",
    "        )\n",
    "        # Via padded[:-1], we remove the extra padded token\n",
    "        # that has been added via the +1 setting in batch_max_length\n",
    "        # (the extra padding token will be relevant in later codes)\n",
    "        inputs = torch.tensor(padded[:-1])\n",
    "        inputs_lst.append(inputs)\n",
    "\n",
    "    # Convert list of inputs to tensor and transfer to target device\n",
    "    inputs_tensor = torch.stack(inputs_lst).to(device)\n",
    "    return inputs_tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "28755ac0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[    0,     1,     2,     3,     4],\n",
      "        [    5,     6, 50256, 50256, 50256],\n",
      "        [    7,     8,     9, 50256, 50256]])\n"
     ]
    }
   ],
   "source": [
    "inputs_1 = [0, 1, 2, 3, 4]\n",
    "inputs_2 = [5, 6]\n",
    "inputs_3 = [7, 8, 9]\n",
    "\n",
    "batch = (\n",
    "    inputs_1,\n",
    "    inputs_2,\n",
    "    inputs_3\n",
    ")\n",
    "\n",
    "print(custom_collate_draft_1(batch))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de045d4f",
   "metadata": {},
   "source": [
    "- 上文我们仅返回了大语言模型的输入数据；然而对于大语言模型训练，我们还需要目标值；\n",
    "- 与预训练大语言模型类似，目标值是将输入序列向右移动一个位置得到的，以便大语言模型学习预测下一个标记；\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a370c316",
   "metadata": {},
   "source": [
    "![](./images/inputs-targets.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "3db320a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "def custom_collate_draft_2(\n",
    "    batch,\n",
    "    pad_token_id=50256,\n",
    "    device=\"cpu\"\n",
    "):\n",
    "    # Find the longest sequence in the batch\n",
    "    batch_max_length = max(len(item)+1 for item in batch)\n",
    "\n",
    "    # Pad and prepare inputs\n",
    "    inputs_lst, targets_lst = [], []\n",
    "\n",
    "    for item in batch:\n",
    "        new_item = item.copy()\n",
    "        # Add an <|endoftext|> token\n",
    "        new_item += [pad_token_id]\n",
    "        # Pad sequences to max_length\n",
    "        padded = (\n",
    "            new_item + [pad_token_id] *\n",
    "            (batch_max_length - len(new_item))\n",
    "        )\n",
    "        inputs = torch.tensor(padded[:-1])  # Truncate the last token for inputs\n",
    "        targets = torch.tensor(padded[1:])  # Shift +1 to the right for targets\n",
    "        inputs_lst.append(inputs)\n",
    "        targets_lst.append(targets)\n",
    "\n",
    "    # Convert list of inputs to tensor and transfer to target device\n",
    "    inputs_tensor = torch.stack(inputs_lst).to(device)\n",
    "    targets_tensor = torch.stack(targets_lst).to(device)\n",
    "    return inputs_tensor, targets_tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "6091445e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[    0,     1,     2,     3,     4],\n",
      "        [    5,     6, 50256, 50256, 50256],\n",
      "        [    7,     8,     9, 50256, 50256]])\n",
      "tensor([[    1,     2,     3,     4, 50256],\n",
      "        [    6, 50256, 50256, 50256, 50256],\n",
      "        [    8,     9, 50256, 50256, 50256]])\n"
     ]
    }
   ],
   "source": [
    "inputs, targets = custom_collate_draft_2(batch)\n",
    "print(inputs)\n",
    "print(targets)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17285bfe",
   "metadata": {},
   "source": [
    "![](./images/batching-step-5.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3f4bab1",
   "metadata": {},
   "source": [
    "![](./images/ignore-index.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "8633166a",
   "metadata": {},
   "outputs": [],
   "source": [
    "def custom_collate_fn(\n",
    "    batch,\n",
    "    pad_token_id=50256,\n",
    "    ignore_index=-100,\n",
    "    allowed_max_length=None,\n",
    "    device=\"cpu\"\n",
    "):\n",
    "    # Find the longest sequence in the batch\n",
    "    batch_max_length = max(len(item)+1 for item in batch)\n",
    "\n",
    "    # Pad and prepare inputs and targets\n",
    "    inputs_lst, targets_lst = [], []\n",
    "\n",
    "    for item in batch:\n",
    "        new_item = item.copy()\n",
    "        # Add an <|endoftext|> token\n",
    "        new_item += [pad_token_id]\n",
    "        # Pad sequences to max_length\n",
    "        padded = (\n",
    "            new_item + [pad_token_id] *\n",
    "            (batch_max_length - len(new_item))\n",
    "        )\n",
    "        inputs = torch.tensor(padded[:-1])  # Truncate the last token for inputs\n",
    "        targets = torch.tensor(padded[1:])  # Shift +1 to the right for targets\n",
    "\n",
    "        # New: Replace all but the first padding tokens in targets by ignore_index\n",
    "        mask = targets == pad_token_id\n",
    "        indices = torch.nonzero(mask).squeeze()\n",
    "        if indices.numel() > 1:\n",
    "            targets[indices[1:]] = ignore_index\n",
    "\n",
    "        # New: Optionally truncate to maximum sequence length\n",
    "        if allowed_max_length is not None:\n",
    "            inputs = inputs[:allowed_max_length]\n",
    "            targets = targets[:allowed_max_length]\n",
    "\n",
    "        inputs_lst.append(inputs)\n",
    "        targets_lst.append(targets)\n",
    "\n",
    "    # Convert list of inputs and targets to tensors and transfer to target device\n",
    "    inputs_tensor = torch.stack(inputs_lst).to(device)\n",
    "    targets_tensor = torch.stack(targets_lst).to(device)\n",
    "\n",
    "    return inputs_tensor, targets_tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "fc1f2c57",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[    0,     1,     2,     3,     4],\n",
      "        [    5,     6, 50256, 50256, 50256],\n",
      "        [    7,     8,     9, 50256, 50256]])\n",
      "tensor([[    1,     2,     3,     4, 50256],\n",
      "        [    6, 50256,  -100,  -100,  -100],\n",
      "        [    8,     9, 50256,  -100,  -100]])\n"
     ]
    }
   ],
   "source": [
    "inputs, targets = custom_collate_fn(batch)\n",
    "print(inputs)\n",
    "print(targets)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5dc993e",
   "metadata": {},
   "source": [
    "- 我们通过具体示例来理解这种用-100替代的作用机制；\n",
    "- 为便于说明，假设存在一个包含0和1两个类别的简单分类任务，类似于第6章的场景；\n",
    "- 当模型最后一层输出如下逻辑值时，我们按以下方式计算损失值；\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "497d9726",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(1.1269)\n"
     ]
    }
   ],
   "source": [
    "logits_1 = torch.tensor(\n",
    "    [[-1.0, 1.0],  # 1st training example\n",
    "     [-0.5, 1.5]]  # 2nd training example\n",
    ")\n",
    "targets_1 = torch.tensor([0, 1])\n",
    "\n",
    "\n",
    "loss_1 = torch.nn.functional.cross_entropy(logits_1, targets_1)\n",
    "print(loss_1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "fb91dfe1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.7936)\n"
     ]
    }
   ],
   "source": [
    "logits_2 = torch.tensor(\n",
    "    [[-1.0, 1.0],\n",
    "     [-0.5, 1.5],\n",
    "     [-0.5, 1.5]]  # New 3rd training example\n",
    ")\n",
    "targets_2 = torch.tensor([0, 1, 1])\n",
    "\n",
    "loss_2 = torch.nn.functional.cross_entropy(logits_2, targets_2)\n",
    "print(loss_2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "7519ad67",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(1.1269)\n",
      "loss_1 == loss_3: tensor(True)\n"
     ]
    }
   ],
   "source": [
    "targets_3 = torch.tensor([0, 1, -100])\n",
    "\n",
    "loss_3 = torch.nn.functional.cross_entropy(logits_2, targets_3)\n",
    "print(loss_3)\n",
    "print(\"loss_1 == loss_3:\", loss_1 == loss_3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09661483",
   "metadata": {},
   "source": [
    "- 由此可见，这3个训练样本产生的损失值与之前2个训练样本计算的损失值相同，这意味着交叉熵损失函数忽略了标签为-100的训练样本；\n",
    "- 默认情况下，PyTorch设置了cross_entropy(..., ignore_index=-100)参数来忽略对应-100标签的样本；\n",
    "- 利用这个-100ignore_index参数，我们可以忽略用于填充训练样本至等长的额外文本结束（填充）标记；\n",
    "- 但我们不应忽略第一个文本结束（填充）标记实例（50256），因为它可以向大语言模型传递响应已完成的信号；\n",
    "- 实践中通常也会掩码对应指令部分的目标标记ID，如下图所示；\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a69ea801",
   "metadata": {},
   "source": [
    "![](./images/mask-instructions.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a11dacd8",
   "metadata": {},
   "source": [
    "## 构建Dataloader"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8303bc2",
   "metadata": {},
   "source": [
    "![](./images/chapter-overview-3.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2970a72e",
   "metadata": {},
   "source": [
    "- 先前custom_collate_fn函数的另一处细节优化是：我们现在直接将数据移至目标设备（如GPU），而非在主训练循环中执行此操作。这种做法能提升效率，因为当custom_collate_fn作为数据加载器组成部分时，设备转移操作可作为后台进程执行；\n",
    "- 通过Python标准库functools中的partial函数，我们创建了一个新函数，该函数已预先填充了原函数的device参数；\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "37e6c039",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Device: cpu\n"
     ]
    }
   ],
   "source": [
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "print(\"Device:\", device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "a92094b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "from functools import partial\n",
    "\n",
    "customized_collate_fn = partial(\n",
    "    custom_collate_fn,\n",
    "    device=device,\n",
    "    allowed_max_length=1024\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "231e44ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import DataLoader\n",
    "\n",
    "\n",
    "num_workers = 0\n",
    "batch_size = 8\n",
    "\n",
    "torch.manual_seed(123)\n",
    "\n",
    "train_dataset = InstructionDataset(train_data, tokenizer)\n",
    "train_loader = DataLoader(\n",
    "    train_dataset,\n",
    "    batch_size=batch_size,\n",
    "    collate_fn=customized_collate_fn,\n",
    "    shuffle=True,\n",
    "    drop_last=True,\n",
    "    num_workers=num_workers\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "79403c10",
   "metadata": {},
   "outputs": [],
   "source": [
    "val_dataset = InstructionDataset(val_data, tokenizer)\n",
    "val_loader = DataLoader(\n",
    "    val_dataset,\n",
    "    batch_size=batch_size,\n",
    "    collate_fn=customized_collate_fn,\n",
    "    shuffle=False,\n",
    "    drop_last=False,\n",
    "    num_workers=num_workers\n",
    ")\n",
    "\n",
    "test_dataset = InstructionDataset(test_data, tokenizer)\n",
    "test_loader = DataLoader(\n",
    "    test_dataset,\n",
    "    batch_size=batch_size,\n",
    "    collate_fn=customized_collate_fn,\n",
    "    shuffle=False,\n",
    "    drop_last=False,\n",
    "    num_workers=num_workers\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "a504253f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train loader:\n",
      "torch.Size([8, 61]) torch.Size([8, 61])\n",
      "torch.Size([8, 76]) torch.Size([8, 76])\n",
      "torch.Size([8, 73]) torch.Size([8, 73])\n",
      "torch.Size([8, 68]) torch.Size([8, 68])\n",
      "torch.Size([8, 65]) torch.Size([8, 65])\n",
      "torch.Size([8, 72]) torch.Size([8, 72])\n",
      "torch.Size([8, 80]) torch.Size([8, 80])\n",
      "torch.Size([8, 67]) torch.Size([8, 67])\n",
      "torch.Size([8, 62]) torch.Size([8, 62])\n",
      "torch.Size([8, 75]) torch.Size([8, 75])\n",
      "torch.Size([8, 62]) torch.Size([8, 62])\n",
      "torch.Size([8, 68]) torch.Size([8, 68])\n",
      "torch.Size([8, 67]) torch.Size([8, 67])\n",
      "torch.Size([8, 77]) torch.Size([8, 77])\n",
      "torch.Size([8, 69]) torch.Size([8, 69])\n",
      "torch.Size([8, 79]) torch.Size([8, 79])\n",
      "torch.Size([8, 71]) torch.Size([8, 71])\n",
      "torch.Size([8, 66]) torch.Size([8, 66])\n",
      "torch.Size([8, 83]) torch.Size([8, 83])\n",
      "torch.Size([8, 68]) torch.Size([8, 68])\n",
      "torch.Size([8, 80]) torch.Size([8, 80])\n",
      "torch.Size([8, 71]) torch.Size([8, 71])\n",
      "torch.Size([8, 69]) torch.Size([8, 69])\n",
      "torch.Size([8, 65]) torch.Size([8, 65])\n",
      "torch.Size([8, 68]) torch.Size([8, 68])\n",
      "torch.Size([8, 60]) torch.Size([8, 60])\n",
      "torch.Size([8, 59]) torch.Size([8, 59])\n",
      "torch.Size([8, 69]) torch.Size([8, 69])\n",
      "torch.Size([8, 63]) torch.Size([8, 63])\n",
      "torch.Size([8, 65]) torch.Size([8, 65])\n",
      "torch.Size([8, 76]) torch.Size([8, 76])\n",
      "torch.Size([8, 66]) torch.Size([8, 66])\n",
      "torch.Size([8, 71]) torch.Size([8, 71])\n",
      "torch.Size([8, 91]) torch.Size([8, 91])\n",
      "torch.Size([8, 65]) torch.Size([8, 65])\n",
      "torch.Size([8, 64]) torch.Size([8, 64])\n",
      "torch.Size([8, 67]) torch.Size([8, 67])\n",
      "torch.Size([8, 66]) torch.Size([8, 66])\n",
      "torch.Size([8, 64]) torch.Size([8, 64])\n",
      "torch.Size([8, 65]) torch.Size([8, 65])\n",
      "torch.Size([8, 75]) torch.Size([8, 75])\n",
      "torch.Size([8, 89]) torch.Size([8, 89])\n",
      "torch.Size([8, 59]) torch.Size([8, 59])\n",
      "torch.Size([8, 88]) torch.Size([8, 88])\n",
      "torch.Size([8, 83]) torch.Size([8, 83])\n",
      "torch.Size([8, 83]) torch.Size([8, 83])\n",
      "torch.Size([8, 70]) torch.Size([8, 70])\n",
      "torch.Size([8, 65]) torch.Size([8, 65])\n",
      "torch.Size([8, 74]) torch.Size([8, 74])\n",
      "torch.Size([8, 76]) torch.Size([8, 76])\n",
      "torch.Size([8, 67]) torch.Size([8, 67])\n",
      "torch.Size([8, 75]) torch.Size([8, 75])\n",
      "torch.Size([8, 83]) torch.Size([8, 83])\n",
      "torch.Size([8, 69]) torch.Size([8, 69])\n",
      "torch.Size([8, 67]) torch.Size([8, 67])\n",
      "torch.Size([8, 60]) torch.Size([8, 60])\n",
      "torch.Size([8, 60]) torch.Size([8, 60])\n",
      "torch.Size([8, 66]) torch.Size([8, 66])\n",
      "torch.Size([8, 80]) torch.Size([8, 80])\n",
      "torch.Size([8, 71]) torch.Size([8, 71])\n",
      "torch.Size([8, 61]) torch.Size([8, 61])\n",
      "torch.Size([8, 58]) torch.Size([8, 58])\n",
      "torch.Size([8, 71]) torch.Size([8, 71])\n",
      "torch.Size([8, 67]) torch.Size([8, 67])\n",
      "torch.Size([8, 68]) torch.Size([8, 68])\n",
      "torch.Size([8, 63]) torch.Size([8, 63])\n",
      "torch.Size([8, 87]) torch.Size([8, 87])\n",
      "torch.Size([8, 68]) torch.Size([8, 68])\n",
      "torch.Size([8, 64]) torch.Size([8, 64])\n",
      "torch.Size([8, 68]) torch.Size([8, 68])\n",
      "torch.Size([8, 71]) torch.Size([8, 71])\n",
      "torch.Size([8, 68]) torch.Size([8, 68])\n",
      "torch.Size([8, 71]) torch.Size([8, 71])\n",
      "torch.Size([8, 61]) torch.Size([8, 61])\n",
      "torch.Size([8, 65]) torch.Size([8, 65])\n",
      "torch.Size([8, 67]) torch.Size([8, 67])\n",
      "torch.Size([8, 65]) torch.Size([8, 65])\n",
      "torch.Size([8, 64]) torch.Size([8, 64])\n",
      "torch.Size([8, 60]) torch.Size([8, 60])\n",
      "torch.Size([8, 72]) torch.Size([8, 72])\n",
      "torch.Size([8, 64]) torch.Size([8, 64])\n",
      "torch.Size([8, 70]) torch.Size([8, 70])\n",
      "torch.Size([8, 57]) torch.Size([8, 57])\n",
      "torch.Size([8, 72]) torch.Size([8, 72])\n",
      "torch.Size([8, 64]) torch.Size([8, 64])\n",
      "torch.Size([8, 68]) torch.Size([8, 68])\n",
      "torch.Size([8, 62]) torch.Size([8, 62])\n",
      "torch.Size([8, 74]) torch.Size([8, 74])\n",
      "torch.Size([8, 80]) torch.Size([8, 80])\n",
      "torch.Size([8, 68]) torch.Size([8, 68])\n",
      "torch.Size([8, 70]) torch.Size([8, 70])\n",
      "torch.Size([8, 91]) torch.Size([8, 91])\n",
      "torch.Size([8, 61]) torch.Size([8, 61])\n",
      "torch.Size([8, 66]) torch.Size([8, 66])\n",
      "torch.Size([8, 80]) torch.Size([8, 80])\n",
      "torch.Size([8, 81]) torch.Size([8, 81])\n",
      "torch.Size([8, 74]) torch.Size([8, 74])\n",
      "torch.Size([8, 82]) torch.Size([8, 82])\n",
      "torch.Size([8, 63]) torch.Size([8, 63])\n",
      "torch.Size([8, 83]) torch.Size([8, 83])\n",
      "torch.Size([8, 68]) torch.Size([8, 68])\n",
      "torch.Size([8, 67]) torch.Size([8, 67])\n",
      "torch.Size([8, 77]) torch.Size([8, 77])\n",
      "torch.Size([8, 91]) torch.Size([8, 91])\n",
      "torch.Size([8, 64]) torch.Size([8, 64])\n",
      "torch.Size([8, 61]) torch.Size([8, 61])\n",
      "torch.Size([8, 75]) torch.Size([8, 75])\n",
      "torch.Size([8, 64]) torch.Size([8, 64])\n",
      "torch.Size([8, 66]) torch.Size([8, 66])\n",
      "torch.Size([8, 78]) torch.Size([8, 78])\n",
      "torch.Size([8, 66]) torch.Size([8, 66])\n",
      "torch.Size([8, 64]) torch.Size([8, 64])\n",
      "torch.Size([8, 83]) torch.Size([8, 83])\n",
      "torch.Size([8, 66]) torch.Size([8, 66])\n",
      "torch.Size([8, 74]) torch.Size([8, 74])\n",
      "torch.Size([8, 69]) torch.Size([8, 69])\n"
     ]
    }
   ],
   "source": [
    "print(\"Train loader:\")\n",
    "for inputs, targets in train_loader:\n",
    "    print(inputs.shape, targets.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23819345",
   "metadata": {},
   "source": [
    "- 根据上方输出可见，所有批次均符合预期地保持批次大小为8，但长度各不相同；\n",
    "- 我们还将通过打印inputs批次中第一个训练样本的内容，再次确认输入数据包含对应标记ID 50256的<|endoftext|>填充标记；\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "051b18c1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([21106,   318,   281, 12064,   326,  8477,   257,  4876,    13, 19430,\n",
      "          257,  2882,   326, 20431, 32543,   262,  2581,    13,   198,   198,\n",
      "        21017, 46486,    25,   198, 30003,  6525,   262,  6827,  1262,   257,\n",
      "          985,   576,    13,   198,   198, 21017, 23412,    25,   198,   464,\n",
      "         5156,   318,   845, 13779,    13,   198,   198, 21017, 18261,    25,\n",
      "          198,   464,  5156,   318,   355, 13779,   355,   257,  4936,    13,\n",
      "        50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256])\n"
     ]
    }
   ],
   "source": [
    "print(inputs[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "99563992",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([  318,   281, 12064,   326,  8477,   257,  4876,    13, 19430,   257,\n",
      "         2882,   326, 20431, 32543,   262,  2581,    13,   198,   198, 21017,\n",
      "        46486,    25,   198, 30003,  6525,   262,  6827,  1262,   257,   985,\n",
      "          576,    13,   198,   198, 21017, 23412,    25,   198,   464,  5156,\n",
      "          318,   845, 13779,    13,   198,   198, 21017, 18261,    25,   198,\n",
      "          464,  5156,   318,   355, 13779,   355,   257,  4936,    13, 50256,\n",
      "         -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100])\n"
     ]
    }
   ],
   "source": [
    "print(targets[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40a7d5ff",
   "metadata": {},
   "source": [
    "## 加载LLM模型"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10a939bf",
   "metadata": {},
   "source": [
    "![](./images/chapter-overview-4.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "21f01b37",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2025-10-07 10:32:13.462513: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:467] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
      "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n",
      "E0000 00:00:1759804333.537687    7547 cuda_dnn.cc:8579] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
      "E0000 00:00:1759804333.558740    7547 cuda_blas.cc:1407] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
      "W0000 00:00:1759804333.716186    7547 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.\n",
      "W0000 00:00:1759804333.716211    7547 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.\n",
      "W0000 00:00:1759804333.716213    7547 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.\n",
      "W0000 00:00:1759804333.716214    7547 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.\n",
      "2025-10-07 10:32:13.734018: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
      "To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "File already exists and is up-to-date: gpt2/124M/checkpoint\n",
      "File already exists and is up-to-date: gpt2/124M/encoder.json\n",
      "File already exists and is up-to-date: gpt2/124M/hparams.json\n",
      "File already exists and is up-to-date: gpt2/124M/model.ckpt.data-00000-of-00001\n",
      "File already exists and is up-to-date: gpt2/124M/model.ckpt.index\n",
      "File already exists and is up-to-date: gpt2/124M/model.ckpt.meta\n",
      "File already exists and is up-to-date: gpt2/124M/vocab.bpe\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2025-10-07 10:32:22.777183: W external/local_xla/xla/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory.\n"
     ]
    }
   ],
   "source": [
    "from gpt_download import download_and_load_gpt2\n",
    "from previous_chapters import GPTModel, load_weights_into_gpt\n",
    "\n",
    "BASE_CONFIG = {\n",
    "    \"vocab_size\": 50257,     # Vocabulary size\n",
    "    \"context_length\": 1024,  # Context length\n",
    "    \"drop_rate\": 0.0,        # Dropout rate\n",
    "    \"qkv_bias\": True         # Query-key-value bias\n",
    "}\n",
    "\n",
    "model_configs = {\n",
    "    \"gpt2-small (124M)\": {\"emb_dim\": 768, \"n_layers\": 12, \"n_heads\": 12},\n",
    "    \"gpt2-medium (355M)\": {\"emb_dim\": 1024, \"n_layers\": 24, \"n_heads\": 16},\n",
    "    \"gpt2-large (774M)\": {\"emb_dim\": 1280, \"n_layers\": 36, \"n_heads\": 20},\n",
    "    \"gpt2-xl (1558M)\": {\"emb_dim\": 1600, \"n_layers\": 48, \"n_heads\": 25},\n",
    "}\n",
    "\n",
    "CHOOSE_MODEL = \"gpt2-small (124M)\"\n",
    "\n",
    "BASE_CONFIG.update(model_configs[CHOOSE_MODEL])\n",
    "\n",
    "model_size = CHOOSE_MODEL.split(\" \")[-1].lstrip(\"(\").rstrip(\")\")\n",
    "settings, params = download_and_load_gpt2(\n",
    "    model_size=model_size,\n",
    "    models_dir=\"gpt2\"\n",
    ")\n",
    "\n",
    "model = GPTModel(BASE_CONFIG)\n",
    "load_weights_into_gpt(model, params)\n",
    "model.eval();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "93f552dd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n",
      "\n",
      "### Instruction:\n",
      "Convert the active sentence to passive: 'The chef cooks the meal every day.'\n"
     ]
    }
   ],
   "source": [
    "torch.manual_seed(123)\n",
    "\n",
    "input_text = format_input(val_data[0])\n",
    "print(input_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "de48f54e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from previous_chapters import (\n",
    "    generate,\n",
    "    text_to_token_ids,\n",
    "    token_ids_to_text\n",
    ")\n",
    "\n",
    "token_ids = generate(\n",
    "    model=model,\n",
    "    idx=text_to_token_ids(input_text, tokenizer),\n",
    "    max_new_tokens=35,\n",
    "    context_size=BASE_CONFIG[\"context_length\"],\n",
    "    eos_id=50256,\n",
    ")\n",
    "generated_text = token_ids_to_text(token_ids, tokenizer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "61416708",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "### Instruction:\n",
      "\n",
      "Convert the active sentence to passive: 'The chef cooks the meal every day.'\n",
      "\n",
      "### Instruction:\n",
      "\n",
      "Convert the active\n"
     ]
    }
   ],
   "source": [
    "response_text = (\n",
    "    generated_text[len(input_text):]\n",
    "    .replace(\"### Response:\", \"\")\n",
    "    .strip()\n",
    ")\n",
    "print(response_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46997d23",
   "metadata": {},
   "source": [
    "## 微调模型"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b62321da",
   "metadata": {},
   "source": [
    "![](./images/chapter-overview-5.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0d24001",
   "metadata": {},
   "outputs": [],
   "source": [
    "from previous_chapters import (\n",
    "    calc_loss_loader,\n",
    "    train_model_simple\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a8aa2877",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.to(device)\n",
    "\n",
    "torch.manual_seed(123)\n",
    "\n",
    "with torch.no_grad():\n",
    "    train_loss = calc_loss_loader(train_loader, model, device, num_batches=5)\n",
    "    val_loss = calc_loss_loader(val_loader, model, device, num_batches=5)\n",
    "\n",
    "print(\"Training loss:\", train_loss)\n",
    "print(\"Validation loss:\", val_loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "672206ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "start_time = time.time()\n",
    "\n",
    "torch.manual_seed(123)\n",
    "\n",
    "optimizer = torch.optim.AdamW(model.parameters(), lr=0.00005, weight_decay=0.1)\n",
    "\n",
    "num_epochs = 2\n",
    "\n",
    "train_losses, val_losses, tokens_seen = train_model_simple(\n",
    "    model, train_loader, val_loader, optimizer, device,\n",
    "    num_epochs=num_epochs, eval_freq=5, eval_iter=5,\n",
    "    start_context=format_input(val_data[0]), tokenizer=tokenizer\n",
    ")\n",
    "\n",
    "end_time = time.time()\n",
    "execution_time_minutes = (end_time - start_time) / 60\n",
    "print(f\"Training completed in {execution_time_minutes:.2f} minutes.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "766ea58f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from previous_chapters import plot_losses\n",
    "# Alternatively:\n",
    "# from llms_from_scratch.ch05 import plot_losses\n",
    "\n",
    "epochs_tensor = torch.linspace(0, num_epochs, len(train_losses))\n",
    "plot_losses(epochs_tensor, tokens_seen, train_losses, val_losses)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "739e7b2f",
   "metadata": {},
   "source": [
    "## 提取和保存响应"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c13ac372",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(123)\n",
    "\n",
    "\n",
    "for entry in test_data[:3]:\n",
    "\n",
    "    input_text = format_input(entry)\n",
    "\n",
    "    token_ids = generate(\n",
    "        model=model,\n",
    "        idx=text_to_token_ids(input_text, tokenizer).to(device),\n",
    "        max_new_tokens=256,\n",
    "        context_size=BASE_CONFIG[\"context_length\"],\n",
    "        eos_id=50256\n",
    "    )\n",
    "    generated_text = token_ids_to_text(token_ids, tokenizer)\n",
    "    response_text = (\n",
    "        generated_text[len(input_text):]\n",
    "        .replace(\"### Response:\", \"\")\n",
    "        .strip()\n",
    ")\n",
    "\n",
    "    print(input_text)\n",
    "    print(f\"\\nCorrect response:\\n>> {entry['output']}\")\n",
    "    print(f\"\\nModel response:\\n>> {response_text.strip()}\")\n",
    "    print(\"-------------------------------------\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "76a0d9a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(123)\n",
    "\n",
    "\n",
    "for entry in test_data[:3]:\n",
    "\n",
    "    input_text = format_input(entry)\n",
    "\n",
    "    token_ids = generate(\n",
    "        model=model,\n",
    "        idx=text_to_token_ids(input_text, tokenizer).to(device),\n",
    "        max_new_tokens=256,\n",
    "        context_size=BASE_CONFIG[\"context_length\"],\n",
    "        eos_id=50256\n",
    "    )\n",
    "    generated_text = token_ids_to_text(token_ids, tokenizer)\n",
    "    response_text = (\n",
    "        generated_text[len(input_text):]\n",
    "        .replace(\"### Response:\", \"\")\n",
    "        .strip()\n",
    ")\n",
    "\n",
    "    print(input_text)\n",
    "    print(f\"\\nCorrect response:\\n>> {entry['output']}\")\n",
    "    print(f\"\\nModel response:\\n>> {response_text.strip()}\")\n",
    "    print(\"-------------------------------------\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60e02208",
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "\n",
    "\n",
    "file_name = f\"{re.sub(r'[ ()]', '', CHOOSE_MODEL) }-sft.pth\"\n",
    "torch.save(model.state_dict(), file_name)\n",
    "print(f\"Model saved as {file_name}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69985bbf",
   "metadata": {},
   "source": [
    "## 评估LLM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "487fb556",
   "metadata": {},
   "outputs": [],
   "source": [
    "import psutil\n",
    "\n",
    "def check_if_running(process_name):\n",
    "    running = False\n",
    "    for proc in psutil.process_iter([\"name\"]):\n",
    "        if process_name in proc.info[\"name\"]:\n",
    "            running = True\n",
    "            break\n",
    "    return running\n",
    "\n",
    "ollama_running = check_if_running(\"ollama\")\n",
    "\n",
    "if not ollama_running:\n",
    "    raise RuntimeError(\"Ollama not running. Launch ollama before proceeding.\")\n",
    "print(\"Ollama running:\", check_if_running(\"ollama\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "303efeb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# This cell is optional; it allows you to restart the notebook\n",
    "# and only run section 7.7 without rerunning any of the previous code\n",
    "import json\n",
    "from tqdm import tqdm\n",
    "\n",
    "file_path = \"instruction-data-with-response.json\"\n",
    "\n",
    "with open(file_path, \"r\") as file:\n",
    "    test_data = json.load(file)\n",
    "\n",
    "\n",
    "def format_input(entry):\n",
    "    instruction_text = (\n",
    "        f\"Below is an instruction that describes a task. \"\n",
    "        f\"Write a response that appropriately completes the request.\"\n",
    "        f\"\\n\\n### Instruction:\\n{entry['instruction']}\"\n",
    "    )\n",
    "\n",
    "    input_text = f\"\\n\\n### Input:\\n{entry['input']}\" if entry[\"input\"] else \"\"\n",
    "\n",
    "    return instruction_text + input_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d02e0a10",
   "metadata": {},
   "outputs": [],
   "source": [
    "import urllib.request\n",
    "\n",
    "def query_model(\n",
    "    prompt,\n",
    "    model=\"llama3\",\n",
    "    url=\"http://localhost:11434/api/chat\"\n",
    "):\n",
    "    # Create the data payload as a dictionary\n",
    "    data = {\n",
    "        \"model\": model,\n",
    "        \"messages\": [\n",
    "            {\"role\": \"user\", \"content\": prompt}\n",
    "        ],\n",
    "        \"options\": {     # Settings below are required for deterministic responses\n",
    "            \"seed\": 123,\n",
    "            \"temperature\": 0,\n",
    "            \"num_ctx\": 2048\n",
    "        }\n",
    "    }\n",
    "\n",
    "\n",
    "    # Convert the dictionary to a JSON formatted string and encode it to bytes\n",
    "    payload = json.dumps(data).encode(\"utf-8\")\n",
    "\n",
    "    # Create a request object, setting the method to POST and adding necessary headers\n",
    "    request = urllib.request.Request(\n",
    "        url,\n",
    "        data=payload,\n",
    "        method=\"POST\"\n",
    "    )\n",
    "    request.add_header(\"Content-Type\", \"application/json\")\n",
    "\n",
    "    # Send the request and capture the response\n",
    "    response_data = \"\"\n",
    "    with urllib.request.urlopen(request) as response:\n",
    "        # Read and decode the response\n",
    "        while True:\n",
    "            line = response.readline().decode(\"utf-8\")\n",
    "            if not line:\n",
    "                break\n",
    "            response_json = json.loads(line)\n",
    "            response_data += response_json[\"message\"][\"content\"]\n",
    "\n",
    "    return response_data\n",
    "\n",
    "\n",
    "model = \"llama3\"\n",
    "result = query_model(\"What do Llamas eat?\", model)\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "887bfe4f",
   "metadata": {},
   "outputs": [],
   "source": [
    "for entry in test_data[:3]:\n",
    "    prompt = (\n",
    "        f\"Given the input `{format_input(entry)}` \"\n",
    "        f\"and correct output `{entry['output']}`, \"\n",
    "        f\"score the model response `{entry['model_response']}`\"\n",
    "        f\" on a scale from 0 to 100, where 100 is the best score. \"\n",
    "    )\n",
    "    print(\"\\nDataset response:\")\n",
    "    print(\">>\", entry['output'])\n",
    "    print(\"\\nModel response:\")\n",
    "    print(\">>\", entry[\"model_response\"])\n",
    "    print(\"\\nScore:\")\n",
    "    print(\">>\", query_model(prompt))\n",
    "    print(\"\\n-------------------------\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92ffcc46",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "build-LLM-from-scratch",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
